Facial Expression Recognition of Speaker Using Vowel Judgment and Features of Thermal Face Image

We developed a method and a system for facial expression recognition of a speaker using vowel judgment and features of a thermal face image. In this study, a speech recognition system saves thermal images at the three timing positions of just before speaking, and just when speaking the phonemes of the first and last vowels. The face direction is estimated to select front-view faces as the targets of facial expression recognition using thermal image processing. Two-dimensional discrete cosine transformation is performed for transforming the grayscale values on each block in the focused face parts of an image into their frequency components, which are used to generate feature vectors. We use pattern recognition through our heuristic rules for facial expression recognition. With this method, intentional facial expressions of “angry”, “happy”, “neutral”, “sad”, and “surprised” are discriminable with good recognition accuracy. Based on our method, we have been developing an on-line system for recognizing the facial expression of a speaker using front-view face judgment, vowel judgment, and thermal image processing. Key-Words: Facial Expression Recognition, Thermal Image Processing, Speech Recognition, Vowel Judgment, Speaker, On-line System